Fyffe House is an historic survivor of Kaikoura's whaling past and was built in an area occupied by Maori for more than 900 years. Kaikoura was first occupied by Maori during the 14th and 15th centuries. A further wave of settlement occurred with the arrival in Kaikoura of Waitaha, then Kati Mamoe and then, during the seventeenth century, the Kati Kuri hapu of Kai Tahu. Kaikoura was sold as part of the Wairau Purchase of 1847 by Ngati Toa, who had conquered much of the northern half of the South Island during the 1830s. Kai Tahu disputed Ngati Toa's right to sell this land and in 1856 the government agreed to compensate Kai Tahu, however inadequately, for the land sold by Ngati Toa, which lay between Kaikoura and Kaiapoi. This occurred with the Kaiapoi Purchase, signed in 1857. It was not, however, until 1858 that the government began to negotiate with Kati Kuri over the purchase of the Kaikoura block itself. In 1859 a deed was signed between the Crown and Kati Kuri which saw an estimated 2.8 million acres exchanged for £300 and 5,558 acres of reserves. By this time Pakeha had already established themselves in the area, with whalers settling on the coast in the early 1840s and farmers further inland from the late 1840s and early 1850s.

Whalers first came to New Zealand waters in 1792, initially to hunt sperm whales from their ships. Shore whaling, the killing of whales from small boats launched from the shore, began in the 1820s. Southern right whales were killed and their blubber rended down. The oil produced from that process was used to fuel lamps and as a lubricant. The first shore-based whaling station in Kaikoura was established by Robert Fyffe in 1842. Fyffe had arrived in New Zealand in 1836. He worked for a few years in the Marlborough whaling industry and then opened Waiopuka station, just to the south of Fyffe House, with one of his workmates from Marlborough, John Murray. In 1843 Fyffe had four whale boats and employed 40 men for the whaling season.

It is believed that the first portion of Fyffe House was a two-roomed cottage built for Fyffe's cooper, Thomas Howell. Coopers played an essential role in the whaling industry as they constructed the barrels needed to store and ship the whale oil. Howell's cottage now comprises the eastern wing of Fyffe House and is thought to date from the mid-1840s. This section has French windows from both rooms opening onto the verandah and it rests on whale bone piles, a particularly unusual feature.

Murray opened a second whaling station in Kaikoura at South Bay in 1844 but appears to have had little interest in running either station. By 1845 Fyffe was the sole owner of both stations and these formed the social and commercial centre of Kaikoura until 1867. Fyffe's production of whale oil began to decline from 1847 and he turned to farming, as did many other whalers, and was one of earliest Pakeha to gain a pasturage licence in the area. (By 1850 the right whale had been hunted almost to extinction.)

With the death of Fyffe in 1854 his cousin George Fyffe bought the whaling equipment and 'the right to whale' from Fyfe's estate, and leased Fyfe's Kahu Tara run. Despite his subsequent purchase of 7,000 acres of Kahu Tara George appears to have lived mainly at Fyffe House, which he expanded during the late 1850s and early 1860s. When he was excavating the foundations for his store he discovered a koiwi (human skeleton), two toki (axes) and the largest moa egg ever found. A foot long (30.5 centimetres), the egg now resides in the Museum of New Zealand/Te Papa Tongarewa in Wellington.

The kitchen wing was the second part of Fyffe House to be constructed and it is possible that it was first built as a separate structure and subsequently connected to the remainder of the house. It is thought that the final two-storeyed wing, built to the west of the cooper's wing and running perpendicular to it, was built in preparation for George's marriage to Catherine Douglas Fyfe in 1860. (It is possible that they were related in some way: 'Fyfe' is the spelling Catherine used in her diary.)

George died in 1867, having fallen off the jetty near his house 'while in an unsound state of mind'. Catherine died the following year and, as there were no children, all of their property was sold. Fyffe House was bought by Joseph Wilkinson Goodall (1835-1911), who transferred to Kaikoura as the district constable in 1865. He later became the Clerk of the Court and Registrar of Births, Deaths and Marriages, as well as holding a number of other public service positions. After his marriage in 1876 he became the manager of the Kaikoura wharf, which was constructed on the site of the old jetty, just down the road from Fyffe House.

The Goodalls made few changes to Fyffe House itself, apart from enclosing part of the verandah to form an office. It seems likely that it was the Goodalls who first established a flower garden in the space between the two wings of the building, and who wallpapered the interior for the first time. After Goodall's death in 1911 the land surrounding the house was subdivided into five sections and the section on which Fyffe House stands, was sold to James Johnston in 1920. Johnston passed the house onto his son, James Allen (Jimmy) Johnston in 1922. Previously a whaler, Jimmy was a Kaikoura County Councillor, contractor for roads and bridges and was involved in many local organisations. He did not live in Fyffe House, which was occupied by Joseph and Maud Low and their eight children. The house was sold by Jimmy Johnston in 1935 to Joseph Soutar, a nephew of Joseph Low. Maud Low (also known as Granny Low) continued to live in Fyffe House, while Soutar lived in a small bach on the property. Soutar left Fyffe House to Maud Low upon his death in 1943.

Maud died in 1951 and left Fyffe House to her youngest son, George. George made few changes to the house, closing off the earliest wing, and using much of the house for storage. After he retired from fishing he kept his boat in the parlour for many years. George bequeathed the house to the New Zealand Historic Places Trust Pouhere Taonga. After his death in 1980 the Trust repaired the house and some years later repainted the exterior in its original pink. The cooper's wing was repaired and redecorated as quarters for curators. The sections of land surrounding the house were sold.

Fyffe House is currently open to the public as a house museum. It rests on a site that is historically important to both Maori and Pakeha. The earliest section of the house was built in the mid-1840s and is Kaikoura's oldest surviving building. Fyffe House is important as a survivor of the early whaling industry in New Zealand, and as a representative of many other aspects of South Island history, including moa hunting, kumara gardening, fishing, farming, port activities, family life and tourism.